VisFusion: Visibility-aware Online 3D Scene Reconstruction from Videos

We propose VisFusion, a visibility-aware online 3D scene reconstruction approach from posed monocular videos. In particular, we aim to reconstruct the scene from volumetric features. Unlike previous reconstruction methods which aggregate features for each voxel from input views without considering its visibility, we aim to improve the feature fusion by explicitly inferring its visibility from a similarity matrix, computed from its projected features in each image pair. Following previous works, our model is a coarse-to-fine pipeline including a volume sparsification process. Different from their works which sparsify voxels globally with a fixed occupancy threshold, we perform the sparsification on a local feature volume along each visual ray to preserve at least one voxel per ray for more fine details. The sparse local volume is then fused with a global one for online reconstruction. We further propose to predict TSDF in a coarse-to-fine manner by learning its residuals across scales leading to better TSDF predictions. Experimental results on benchmarks show that our method can achieve superior performance with more scene details. Code is available at: https://github.com/huiyu-gao/VisFusionComment: CVPR 202

Model optimization for fast discrimination of transgenic soybeans using near-infrared spectroscopy

Objective Near-infrared (NIR) spectroscopy and partial least squares-discriminant analysis (PLS-DA) were applied to discriminate soybean samples as being transgenic or non-transgenic. The rapid discrimination models for transgenic soybean were established, and the optimal model was selected. Methods Principal component analysis (PCA) was used to extract relevant features from the spectral data and remove anomalous samples. In experimental studies, 94 samples were used to build models and 41 samples were used as the validation to evaluate the performance of the developed models. The effects of sample morphology (intact or ground), wavelength range and spectral pretreatment method on the correctness of the model were discussed. Results Models for intact soybean samples obtain better judgment performance than models for ground samples. The best discriminant model for intact soybean samples possessed both 100.00% discriminant correct rate in calibration and validation sets at 9 403-5 438 cm-1 using second derivative (2nd). The best discriminant model for ground soybean samples also achieved both 100.00% discriminant correct rate in calibration and validation sets at 7 505-4 597 cm-1 using standard normal variate plus first derivative (SNV+1st). Conclusion By selecting sample morphology, wavelength range and spectral pretreatment method, the discrimination model can be optimized and the discriminant correct rate can be significantly improved

Attention Graph Convolution Network for Image Segmentation in Big SAR Imagery Data

The recent emergence of high-resolution Synthetic Aperture Radar (SAR) images leads to massive amounts of data. In order to segment these big remotely sensed data in an acceptable time frame, more and more segmentation algorithms based on deep learning attempt to take superpixels as processing units. However, the over-segmented images become non-Euclidean structure data that traditional deep Convolutional Neural Networks (CNN) cannot directly process. Here, we propose a novel Attention Graph Convolution Network (AGCN) to perform superpixel-wise segmentation in big SAR imagery data. AGCN consists of an attention mechanism layer and Graph Convolution Networks (GCN). GCN can operate on graph-structure data by generalizing convolutions to the graph domain and have been successfully applied in tasks such as node classification. The attention mechanism layer is introduced to guide the graph convolution layers to focus on the most relevant nodes in order to make decisions by specifying different coefficients to different nodes in a neighbourhood. The attention layer is located before the convolution layers, and noisy information from the neighbouring nodes has less negative influence on the attention coefficients. Quantified experiments on two airborne SAR image datasets prove that the proposed method outperforms the other state-of-the-art segmentation approaches. Its computation time is also far less than the current mainstream pixel-level semantic segmentation networks

Weakly Supervised Segmentation of SAR Imagery Using Superpixel and Hierarchically Adversarial CRF

Synthetic aperture radar (SAR) image segmentation aims at generating homogeneous regions from a pixel-based image and is the basis of image interpretation. However, most of the existing segmentation methods usually neglect the appearance and spatial consistency during feature extraction and also require a large number of training data. In addition, pixel-based processing cannot meet the real time requirement. We hereby present a weakly supervised algorithm to perform the task of segmentation for high-resolution SAR images. For effective segmentation, the input image is first over-segmented into a set of primitive superpixels. This algorithm combines hierarchical conditional generative adversarial nets (CGAN) and conditional random fields (CRF). The CGAN-based networks can leverage abundant unlabeled data learning parameters, reducing their reliance on the labeled samples. In order to preserve neighborhood consistency in the feature extraction stage, the hierarchical CGAN is composed of two sub-networks, which are employed to extract the information of the central superpixels and the corresponding background superpixels, respectively. Afterwards, CRF is utilized to perform label optimization using the concatenated features. Quantified experiments on an airborne SAR image dataset prove that the proposed method can effectively learn feature representations and achieve competitive accuracy to the state-of-the-art segmentation approaches. More specifically, our algorithm has a higher Cohen’s kappa coefficient and overall accuracy. Its computation time is less than the current mainstream pixel-level semantic segmentation networks

Integrated GANs: Semi-Supervised SAR Target Recognition

With the advantage of working in all weathers and all days, synthetic aperture radar (SAR) imaging systems have a great application value. As an efficient image generation and recognition model, generative adversarial networks (GANs) have been applied to SAR image analysis and achieved promising performance. However, the cost of labeling a large number of SAR images limits the performance of the developed approaches and aggravates the mode collapsing problem. This paper presents a novel approach namely Integrated GANs (I-GAN), which consists of a conditional GANs, an unconditional GANs and a classifier, to achieve semi-supervised generation and recognition simultaneously. The unconditional GANs assist the conditional GANs to increase the diversity of the generated images. A co-training method for the conditional GANs and the classifier is proposed to enrich the training samples. Since our model is capable of representing training images with rich characteristics, the classifier can achieve better recognition accuracy. Experiments on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset proves that our method achieves better results in accuracy when labeled samples are insufficient, compared against other state-of-the-art techniques

RoadSeg-CD: A Network With Connectivity Array and Direction Map for Road Extraction From SAR Images

Road extraction from synthetic aperture radar (SAR) images has attracted much attention in the field of remote sensing image processing. General road extraction algorithms, affected by shadows of buildings and trees, are prone to producing fragmented road segments. To improve the accuracy and completeness of road extraction, we propose a neural network-based algorithm, which takes the connectivity and direction features of roads into consideration, named RoadSeg-CD. It consists of two branches: one is the main branch for road segmentation; the other is the auxiliary branch for learning road directions. In the main branch, a connectivity array is designed to utilize local contextual information and construct a connectivity loss based on the predicted probabilities of neighboring pixels. In the auxiliary branch, we proposed a novel road direction map, which is used for learning the directions of roads. The two branches are connected by specific feature fusion process, and the output from the main branch is taken as the road extraction result. Experiments on real radar images are implemented to validate the effectiveness of our method. The experimental results demonstrate that our method can obtain more continuous and more complete roads than several state-of-the-art road extraction algorithms

Airport Detection in SAR Images via Salient Line Segment Detector and Edge-Oriented Region Growing

Airport detection in synthetic aperture radar (SAR) images has attracted much concern in the field of remote sensing. Affected by other salient objects with geometrical features similar to those of airports, traditional methods often generate false detections. In order to produce the geometrical features of airports and suppress the influence of irrelevant objects, we propose a novel method for airport detection in SAR images. First, a salient line segment detector is constructed to extract salient line segments in the SAR images. Second, we obtain the airport support regions by grouping these line segments according to the commonality of these geometrical features. Finally, we design an edge-oriented region growing (EORG) algorithm, where growing seeds are selected from the airport support regions with the help of edge information in SAR images. Using EORG, the airport region can be mapped by performing region growing with these seeds. We implement experiments on real radar images to validate the effectiveness of our method. The experimental results demonstrate that our method can acquire more accurate locations and contours of airports than several state-of-the-art airport detection algorithms

Anchor-free Convolutional Network with Dense Attention Feature Aggregation for Ship Detection in SAR Images

In recent years, with the improvement of synthetic aperture radar (SAR) imaging resolution, it is urgent to develop methods with higher accuracy and faster speed for ship detection in high-resolution SAR images. Among all kinds of methods, deep-learning-based algorithms bring promising performance due to end-to-end detection and automated feature extraction. However, several challenges still exist: (1) standard deep learning detectors based on anchors have certain unsolved problems, such as tuning of anchor-related parameters, scale-variation and high computational costs. (2) SAR data is huge but the labeled data is relatively small, which may lead to overfitting in training. (3) To improve detection speed, deep learning detectors generally detect targets based on low-resolution features, which may cause missed detections for small targets. In order to address the above problems, an anchor-free convolutional network with dense attention feature aggregation is proposed in this paper. Firstly, we use a lightweight feature extractor to extract multiscale ship features. The inverted residual blocks with depth-wise separable convolution reduce the network parameters and improve the detection speed. Secondly, a novel feature aggregation scheme called dense attention feature aggregation (DAFA) is proposed to obtain a high-resolution feature map with multiscale information. By combining the multiscale features through dense connections and iterative fusions, DAFA improves the generalization performance of the network. In addition, an attention block, namely spatial and channel squeeze and excitation (SCSE) block is embedded in the upsampling process of DAFA to enhance the salient features of the target and suppress the background clutters. Third, an anchor-free detector, which is a center-point-based ship predictor (CSP), is adopted in this paper. CSP regresses the ship centers and ship sizes simultaneously on the high-resolution feature map to implement anchor-free and nonmaximum suppression (NMS)-free ship detection. The experiments on the AirSARShip-1.0 dataset demonstrate the effectiveness of our method. The results show that the proposed method outperforms several mainstream detection algorithms in both accuracy and speed

RoadSeg-CD: A Network With Connectivity Array and Direction Map for Road Extraction From SAR Images

Road extraction from synthetic aperture radar (SAR) images has attracted much attention in the field of remote sensing image processing. General road extraction algorithms, affected by shadows of buildings and trees, are prone to producing fragmented road segments. To improve the accuracy and completeness of road extraction, we propose a neural network-based algorithm, which takes the connectivity and direction features of roads into consideration, named RoadSeg-CD. It consists of two branches: one is the main branch for road segmentation; the other is the auxiliary branch for learning road directions. In the main branch, a connectivity array is designed to utilize local contextual information and construct a connectivity loss based on the predicted probabilities of neighboring pixels. In the auxiliary branch, we proposed a novel road direction map, which is used for learning the directions of roads. The two branches are connected by specific feature fusion process, and the output from the main branch is taken as the road extraction result. Experiments on real radar images are implemented to validate the effectiveness of our method. The experimental results demonstrate that our method can obtain more continuous and more complete roads than several state-of-the-art road extraction algorithms